Skip to content

Update s3 scan ack timeout and scan interval in rds transform template#5820

Merged
oeyh merged 1 commit into
opensearch-project:mainfrom
oeyh:rds/s3-scan-delay-fix
Jun 25, 2025
Merged

Update s3 scan ack timeout and scan interval in rds transform template#5820
oeyh merged 1 commit into
opensearch-project:mainfrom
oeyh:rds/s3-scan-delay-fix

Conversation

@oeyh
Copy link
Copy Markdown
Collaborator

@oeyh oeyh commented Jun 25, 2025

Description

Noticed long delay in s3 scan in RDS source, similar to what is described in #4988. Update s3 scan ack timeout to 10min from the default 2h and scan interval to 20s in rds transform template, same as documentdb template

Issues Resolved

N/A

Check List

  • New functionality includes testing.
  • New functionality has a documentation issue. Please link to it in this PR.
    • New functionality has javadoc added
  • Commits are signed with a real name per the DCO

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Hai Yan <oeyh@amazon.com>
@oeyh oeyh merged commit 7189eda into opensearch-project:main Jun 25, 2025
44 of 46 checks passed
JonahCalvo pushed a commit to JonahCalvo/os-data-prepper that referenced this pull request Jul 17, 2025
…rch-project#5820)

Signed-off-by: Hai Yan <oeyh@amazon.com>
Signed-off-by: Jonah Calvo <caljonah@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants